The present invention relates generally to a system and method for efficiently validating the functionality and performance software applications. In particular, the system and method relates to a multiple test phase integrated process for testing applications, in which the system and method enables the efficient use of test phase deliverables to produce high quality business requirements-based software applications.
Typically web-based software applications are designed to accommodate a multitude of different kinds of business transactions, where each transaction often requires a significant number of functions to be performed in order to complete that transaction. The quality of a software application depends largely on the extensive functionality testing required to validate each function used to execute an on-line transaction and that each on-line transaction is consistent with the business functionality requirements. Software developers recognize that testing software applications is a continuous process that is often time consuming, repetitive and expensive. The testing process is usually divided into test phases which include unit test, system test, integration, user acceptance testing, customer installation and support test phases. Of the various test phases, phases such as the system test phase, which are designed to validate the functionality and performance of a software application, quite often involve the largest number of repetitive tests and tasks.
Repetitive testing and other inefficient testing methods are often the result of having the various phases of test performed by separate and independent organizations that do not communicate with each other. In this type of testing scheme one organization performs the functional tests which involve the creation of requirements-based conditions, test data and manual test scripts. Another organization independently performs automated regression tests which also involve creating requirements-based test conditions, test data and manual test scripts and a third organization conducts performance tests which again similarly requires the creation of requirements-based test conditions, test data and test scripts.
Those skilled in software test have recognized that the use of independent organizations to perform tests such as those in the functional, automated regression and performance test phases is an overall testing method that results a high degree of duplicative testing. Duplicative testing is highly inefficient, costly and time-consuming. These types of inefficiencies are particularly apparent in the related areas of functional, automated regression and performance phase testing.
In addition to cost and time inefficiencies that may arise from a test process that uses independent test organizations, quality issues may also be a concern. A quality issue that emerges from separate functional, automated regression and performance phase testing is the reduced level of consistency or coherence between test results from one phase and another. These inconsistencies often stem from the fact that different test organizations, although operating from the same business requirements, may select varying requirements and functions to test.
Variations in the requirements that are selected for test necessarily causes variations in the test data and scripts that are derived from such requirements. It may be the case that after the three test phases have been completed it is discovered that some functions and business requirements of the application were tested in one phase but not in another. Another factor contributing to the inconsistency of test coverage is that the method of testing a particular function or requirement may vary from organization to organization. As a result, variations in the testing methods between phases and organizations may lead to a reduced level of confidence in the quality of the software application that has been tested, even if that application has passed all of the prescribed tests.
Inefficiencies in current testing methods are not exclusively the result of varying testing methodologies of independent organizations. Test procedure inefficiencies also stem from the current methods or testing tools used to perform the automated regression and performance tests. Standard automated testing tools allow the user to record and playback automatically inputs that are manually entered into the system by an analysts. This method of test is common and may be adequate for relatively small testing jobs but for larger jobs the level of manual repetition is labor and time intensive as well as extremely costly. Similarly, other inefficiencies may be present in current method for conducting performance testing. Performance testing inefficiencies may be due to low test coverage, an inappropriate selection of business functions to be tested and duplicative testing.
For the foregoing reasons, in the software testing industry there remains a need for a requirements-based method of testing applications software that is capable of producing consistent, coherent test results from the functional, automated regression and performance phases of testing.
For the foregoing reasons, there is also a need for an end-to-end integrated method for validating the functionality and performance of software applications that allows deliverables, such as functional and technical conditions and manual test scripts to be passed on from one phase to another when conducting functional, automated regression and performance testing in order to expedite the overall system test cycle, enhance efficiencies, produce consistent results from phase to phase and a high level of confidence in the quality of the software application.
For the foregoing reasons, there is also a need for an organizational structure that promotes the integration of testing teams so that communication is enhanced, information is effectively shared, work duplication minimized and testing costs reduced.
For the foregoing reasons, there is a need for a performance testing process that includes the initial identification of key business transactions and the implementation of script standards that will increase test case coverage, and enhance application quality.
For the forgoing reasons, there is also a need for a functional and automated regression testing method that effectively segments testing tasks within the test phases such that task complexity is efficiently matched to an analyst's level of skill.